1. Field of the Invention
The present invention relates to a fluorescence-endoscope apparatus that acquires a plurality of types of images of fluorescence generated by biological tissue containing a plurality of types of fluorescent components, whose maximum-fluorescence wavelengths are different and whose fluorescence wavelengths overlap in at least parts of the wavelength ranges, and that separately displays the plurality of types of fluorescent components present in the biological tissue using the acquired fluorescence images.
2. Description of Related Art
In molecular imaging diagnosis using fluorescence-endoscope apparatuses, it is effective to perform fluorescence-extraction processing using a so-called UNMIXING technique that removes autofluorescence noise originating from biological subjects (tissue, residues, and so forth), to extract fluorescence coming from fluorescent probes. In order to improve the image quality S/N of the fluorescence image with low light intensity in the fluorescence-extraction processing using the UNMIXING technique, it is effective to allow the exposure time to be changed arbitrarily via a spectral image-acquisition unit provided with an etalon-type tunable light-dispersing element and a sensitive camera, for example.
Conventional fluorescence-endoscope apparatuses using the UNMIXING technique include, for example, the endoscope apparatus described in the following Publication of Japanese Patent No. 2008-43396.
In fluorescence-endoscope apparatuses that capture spectral images of a plurality of types of fluorescence, the longer the exposure time used, the lower the frame rate becomes.
The brightness of each fluorescent component (human tissue, residue, or fluorescence agent) present in the biological tissue of an observation target is not uniform.
Thus, for example, when a darker fluorescent component is present in the biological tissue, it is necessary to increase the exposure time so that the brightness of the fluorescence image of the darker fluorescent component is suitable for observation. However, if the exposure time for the fluorescence images of other fluorescent components present in the biological tissue is increased in a similar manner in accordance with the change made in the exposure time for the fluorescence image of the darker fluorescent component, the frame rate is greatly reduced. When the other fluorescent components present in the biological tissue are excessively bright and if the exposure time is increased further, the brightnesses of the fluorescence images of the excessively-brighter fluorescent components are saturated.
For example, when an excessively bright fluorescent component is present in the biological tissue, it is necessary to reduce the exposure time such that the brightness of the image of the excessively bright fluorescent component is suitable for observation. However, if the exposure time for the fluorescence images of the other fluorescent components present in the biological tissue is reduced in a similar manner in accordance with the change made in the exposure time for the fluorescence image of the excessively bright fluorescent component and if, for example, the other fluorescent components are darker, the fluorescence images of the darker fluorescent components become too dark to be detected.
Therefore, in fluorescence-extraction processing using the UNMIXING technique in the endoscope apparatus described in Publication of Japanese Patent No. 2008-43396, constant UNMIXING coefficients (component ratios of the fluorescent components) are used regardless of the exposure conditions, such as the exposure time etc. Thus, when the fluorescence image is detected by adjusting the exposure conditions, such as the exposure time etc., so that a suitable brightness is achieved for each fluorescence wavelength, the UNMIXING coefficients become unsuitable, and it is sometimes difficult to separate the fluorescent components suitably.
The fluorescence-endoscope apparatus using the conventional UNMIXING technique will be described with reference to an example where a sample containing three types of fluorescent components having different fluorescence wavelengths is subjected to spectroscopy.
FIGS. 15A and 15B are diagrams for explaining three types of fluorescent components 1 to 3 present in a sample, where FIG. 15A is a diagram conceptually showing the distribution of the fluorescent components 1 to 3 in the sample, and FIG. 15B is a diagram showing fluorescence spectra of the fluorescent components 1 to 3. FIGS. 16A to 16C are explanatory diagrams conceptually showing the distributions and brightnesses of spectral images acquired by an image acquisition apparatus through a tunable light-dispersing element, such as an etalon etc., in a fluorescence-endoscope apparatus, where FIG. 16A is a diagram showing a spectral image of the fluorescent component 1 at the maximum-fluorescence wavelength λ1, FIG. 16B is a diagram showing a spectral image of the fluorescent component 2 at the maximum-fluorescence wavelength λ2, and FIG. 16C is a diagram showing a spectral image of the fluorescent component 3 at the maximum-fluorescence wavelength λ3. Iall(λ1), Iall(λ2), and Iall(λ3) are signal intensities that are detected at an arbitrary common pixel Pi in each of the spectral images shown in FIGS. 16A to 16C. FIGS. 17A to 17C are diagrams showing example exposure timings for spectral images of the individual fluorescent components 1 to 3 at the maximum-fluorescence wavelengths λ1 to λ3, where FIG. 17A is a diagram showing transmission-switching timings of the maximum-fluorescence wavelengths λ1 to λ3 of the three types of fluorescent components 1 to 3 by a tunable light-dispersing element, FIG. 17B is a diagram showing image-acquisition timings of the fluorescence wavelengths λ1 to λ3 that are acquired by an image acquisition apparatus, and FIG. 17C is a diagram showing an example of a matrix equation for calculating densities D1 to D3 of the fluorescent components 1 to 3 using the conventional UNMIXING technique in a spectral image in which fluorescence signals generated by the fluorescent components 1 to 3 coexist. In FIGS. 17A to 17C, tλ1 to tλ3 are exposure times at which the individual fluorescence wavelengths λ1 to λ3 are spectrally separated through a tunable light-dispersing element etc., and M1 to M3 are memories that store the data of the respective fluorescence wavelengths λ1 to λ3, which have been spectrally separated through the tunable light-dispersing element etc., acquired by individual image acquisition apparatuses.
Image acquisition using the image acquisition apparatus is performed on a sample, serving as a target, in which, as shown in FIG. 15B, the fluorescent component 1 having the maximum-fluorescence wavelength at the wavelength λ1, the fluorescent component 2 having the maximum-fluorescence wavelength at the wavelength λ2, and the fluorescent component 3 having the maximum-fluorescence wavelength at the wavelength λ3 are individually distributed in the locations shown in FIG. 15A, by transmitting the fluorescence wavelengths λ1 to λ3 generated by the individual fluorescent components 1 to 3 through the tunable light-dispersing element in a time-division manner.
At this time, as shown in FIGS. 16A to 16C, although the intensities of the fluorescence signals from the individual fluorescent components 1 to 3 are different, the spectral images at the individual fluorescence wavelengths are images that contain the fluorescence signals generated by the three types of fluorescent components in a mixed manner. In other words, in these spectral images, fluorescence signals emitted from the fluorescent components 1 to 3 are not separated.
Here, it is assumed that fluorescence-extraction processing using the conventional UNMIXING technique described in Publication of Japanese Patent No. 2008-43396 is performed on a spectral image containing the fluorescence signals from the fluorescent components 1 to 3 in a mixed manner.
In the fluorescence-extraction processing using the conventional UNMIXING technique, the UNMIXING coefficients (the component ratios of the fluorescent components) of the fluorescence spectra of the three types of fluorescent components 1 to 3 present in the sample at the individual normalized densities are stored in advance in a predetermined storage medium.
Then, the UNMIXING coefficients (the component ratios of the fluorescent components) of the fluorescence spectra of the fluorescent components 1 to 3 at the individual normarized densities stored in the predetermined storage medium and the intensities Iall(λ1) to Iall(λ3) in the fluorescence images at the individual fluorescence wavelengths λ1 to λ3 that are detected are used to calculate the densities D1 to D3 of the fluorescent components 1 to 3 based on the matrix equation shown in FIG. 17C.
In this way, in the fluorescence-endoscope apparatus using the conventional UNMIXING technique, it is possible to obtain distribution images of the individual fluorescent components by performing UNMIXING on the spectral images using the spectra of the individual fluorescent components.
FIGS. 18A to 18G are diagrams showing image processing after the UNMIXING processing, where FIG. 18A is a distribution image of the fluorescent component 1 after the UNMIXING processing, FIG. 18B is a diagram showing a state in which a predetermined color is assigned to a distributed region of the fluorescent component 1 in the distribution image in FIG. 18A, FIG. 18C is a distribution image of the fluorescent component 2 after the UNMIXING processing, FIG. 18D is a diagram showing a state in which a predetermined color is assigned to a distributed region of the fluorescent component 2 in the distribution image in FIG. 18C, FIG. 18E is a distribution image of the fluorescent component 3 after the UNMIXING processing, FIG. 18F is a diagram showing a state in which a predetermined color is assigned to a distributed region of the fluorescent component 3 in the distribution image in FIG. 18E, and FIG. 18G is a diagram showing a state in which the distribution images shown in FIGS. 18B, 18D, and 18F are combined into one image.
In accordance with the brightnesses of the individual fluorescent components; their tendency to accumulate in a biological subject, serving as a sample; their transportability to a lesion, serving as a sample; metabolic properties in a biological subject (such as time); time (assessment time) and timing before performing measurement after introducing a fluorophor into the biological subject; and so forth, the fluorescence intensities (brightnesses) at the maximum-fluorescence wavelength are different for every fluorescent component in the sample.
For example, even with the fluorescent components 1 to 3 having the fluorescence spectral properties shown in FIG. 19A, as shown in FIG. 19B, the fluorescence intensity of the fluorescent component 2 at the maximum-fluorescence wavelength λ2 may be different.
In such a case, it is difficult to detect a darker fluorescent component if changing the exposure conditions, including the exposure time etc., such as increasing the exposure time for the fluorescence image of the fluorescent component 2 in FIGS. 19A and 19B, is not performed properly. Because the intensity detected from the darker fluorescent component 2 at the fluorescence wavelength λ2 is weak, the detected signal tends to be affected by noise, and it is difficult to obtain the correct density even when the UNMIXING processing is performed.
However, for example, if the exposure time is evenly increased for the fluorescence images of all fluorescent components at the maximum-fluorescence wavelength in accordance with a suitable adjustment of the exposure time for the fluorescent component that is dark at the maximum-fluorescence wavelength, the frame rate is considerably reduced.
For the fluorescent component that is dark at the maximum-fluorescence wavelength, if only the exposure time for the fluorescence image at the maximum-fluorescence wavelength is increased, then the detection level of the fluorescence wavelengths of the other fluorescent components contained in the image varies, and a divergence from the UNMIXING coefficients (the component ratios of the fluorescent components) occurs, making separation of the fluorescent components difficult.
For example, in contrast to the situation shown in FIGS. 19A and 19B, in the case where the fluorescent component 2 is excessively bright at the maximum-fluorescence wavelength, the brightness of the image of the fluorescent component 2 is saturated if changing the exposure conditions, including the exposure time etc., such as reducing the exposure time for the fluorescence image of the fluorescent component 2, is not performed properly.
However, for example, if the exposure time is evenly decreased for the fluorescence images of all fluorescent components at the maximum-fluorescence wavelength in accordance with a suitable adjustment of the exposure time for the fluorescent component that is bright at the maximum-fluorescence wavelength, then the fluorescence images of the other fluorescent components at the maximum-fluorescence wavelength become darker, making it difficult to detect the darker fluorescent components, or the detected signal tends to be affected by noise, making it difficult to obtain the correct densities even if the UNMIXING processing is performed.
If only the exposure time for the fluorescence image at the maximum-fluorescence wavelength of the fluorescent component that is excessively-bright at the maximum-fluorescence wavelength is reduced, then the detection level of the fluorescence wavelengths of the other fluorescent components present in the image varies, and a divergence from the UNMIXING coefficients (the component ratios of the fluorescent components) occurs, making separation of the fluorescent components difficult.
As shown in FIG. 22B, when only the exposure time for the fluorescence image at the fluorescence wavelength λ2 shown in FIG. 21B is increased, not only the fluorescent component 2, but also the other components, such as the fluorescent component 1 and the fluorescent component 3, are detected at high brightness.
In such a case, even if the UNMIXING processing is performed on the acquired spectral images, the component ratios of the actual fluorescent components become different from the UNMIXING coefficients that have been stored in a memory etc. in advance. Therefore, even if the densities D1 to D3 of the fluorescent components 1 to 3 are obtained using the matrix equation shown in FIG. 17C, to which matrix equation (13) above has been applied, as shown in FIGS. 23B to 23D, it is difficult to acquire distribution images that are separated into the individual fluorescent components.
A fluorescence-endoscope apparatus of one aspect of the present invention is a fluorescence-endoscope apparatus that radiates excitation light onto biological tissue containing a plurality of types of fluorescent components, whose maximum-fluorescence wavelengths are different and whose fluorescence wavelengths overlap in at least parts of the wavelength ranges, that acquires a plurality of types of images of fluorescence generated by the biological tissue, and that displays, in a separated manner, the plurality of types of fluorescent components present in the biological tissue using the acquired fluorescence images, comprising: a light source portion that emits at least one type of excitation light that excites the plurality of types of fluorescent components; a fluorescence image capturing unit that acquires the images of fluorescence generated by the biological tissue for every n types [where, m≦n] of wavelengths λ1 to wavelength λn; a fluorescence-spectrum storage unit that records fluorescence spectra of m types [where, 2≦m] of individual fluorescent component 1 to fluorescent component m present in the biological tissue at normalized densities under reference exposure conditions; a fluorescent-component-density computation unit that obtains densities of the individual fluorescent components present in the biological tissue for all pixels in the fluorescence images by performing computation using the fluorescence spectra at the individual normalized densities of the fluorescent component 1 to fluorescent component m under the reference exposure conditions that are stored in the fluorescence-spectrum storage unit and the fluorescence images for every wavelength λ1 to wavelength λn acquired by the fluorescence image capturing unit; a fluorescence-image combining portion that forms distribution images of the individual fluorescent components on the basis of the density of the individual fluorescent components obtained by the fluorescent-component-density computation unit, assigns predetermined colors corresponding to the individual fluorescent components to the formed distribution images of the individual fluorescent components, and combines the distribution images to which the predetermined colors are assigned into one image; and an image display portion that displays the image that has been combined by the fluorescence-image combining portion, wherein, when a1 (λ1) to a1(λn) to am(λ1) to am(λn) are defined as coefficients at the wavelength λ1 to wavelength λn of the fluorescent component 1 to fluorescent component m at the individual normalized densities under the reference exposure conditions, which are obtained from the fluorescence spectra, stored in the fluorescence-spectrum storage unit, of the fluorescent component 1 to fluorescent component m at the individual normalized densities under the reference exposure conditions, Iall(λ1) to Iall(λn) are defined as intensities of the fluorescence images at the wavelength λ1 to wavelength λn acquired by the fluorescence image capturing unit, and D1 to Dm are defined as the densities of the fluorescent component 1 to fluorescent component m, the fluorescent-component-density computation unit calculates, for all pixels, the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m in each pixel in the fluorescence images using Equation (1) below, and wherein the fluorescent-component-density computation unit: checks if the reference exposure conditions of an exposure condition item have been changed; and if a value of a predetermined exposure condition item has been changed when the fluorescence image at least one wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), changes the coefficients a1 (λx) to am(λx) at the wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions using the ratio of the value of the changed predetermined exposure condition item to the value of the predetermined exposure condition item under the reference exposure conditions when the fluorescence image at the wavelength λx is acquired by the fluorescence image capturing unit:
                    [                  Expression          ⁢                                          ⁢          5                ]                                                                      (                                                                      D                  ⁢                                                                          ⁢                  1                                                                                    ⋮                                                                                      D                  ⁢                                                                          ⁢                  m                                                              )                =                                            (                                                                                          a                      ⁢                                                                                          ⁢                      1                      ⁢                                              (                        λ1                        )                                                                                                  …                                                                              a                      ⁢                                                                                          ⁢                                              m                        ⁡                                                  (                                                      λ                            ⁢                                                                                                                  ⁢                            1                                                    )                                                                                                                                                          ⋮                                                        ⋮                                                        ⋮                                                                                                              a                      ⁢                                                                                          ⁢                      1                      ⁢                                              (                                                  λ                          ⁢                                                                                                          ⁢                          n                                                )                                                                                                  …                                                                              a                      ⁢                                                                                          ⁢                                              m                        ⁡                                                  (                                                      λ                            ⁢                                                                                                                  ⁢                            n                                                    )                                                                                                                                )                                      -              1                                ⁢                      (                                                                                                      I                      all                                        ⁡                                          (                      λ1                      )                                                                                                                    ⋮                                                                                                                        I                      all                                        ⁡                                          (                                              λ                        ⁢                                                                                                  ⁢                        n                                            )                                                                                            )                                                      ⁢                  (          1          )                    Equation (1).
In the above-mentioned fluorescence-endoscope apparatus, if the exposure time has been changed when the fluorescence image at the predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the predetermined wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed exposure time to the exposure time under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if the exposure times have been changed while keeping the frame rate constant when the fluorescence images at the individual wavelengths of the wavelength λ1 to wavelength λn are acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λ1) to am(λ1) to a1 (λn) to am(λn) at individual wavelengths among the wavelength λ1 to wavelength λn of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed exposure time to the exposure time under the reference exposure conditions when the fluorescence images at the individual wavelengths of the wavelength λ1 to wavelength λn are acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if the intensity of the excitation light that excites the predetermined wavelength λx has been changed when the fluorescence image at the predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed intensity of the excitation light that excites the predetermined wavelength λx to the intensity of the excitation light that excites the predetermined wavelength λx under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if the intensities of the excitation light beams that excite the individual wavelengths have been changed when the fluorescence images at the individual wavelengths of the wavelength λ1 to wavelength λn are acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λ1) to am(λ1) to a1 (λn) to am(λn) at individual wavelengths among the wavelengths λ1 to λn of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratios of the changed intensities of the excitation light beams that excite the individual wavelengths to the intensities of the excitation light beams that excite the individual wavelengths under the reference exposure conditions when the fluorescence images at the individual wavelengths of the wavelength λ1 to wavelength λn are acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if the excitation time of the excitation light that excites the predetermined wavelength λx has been changed when the fluorescence image at predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed excitation time of the excitation light that excites the predetermined wavelength λx to the excitation time of the excitation light that excites the predetermined wavelength λx under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if the intensity and the excitation time of the excitation light that excites the predetermined wavelength λx have been changed when the fluorescence image at predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratios of the changed intensity and the changed excitation time of the excitation light that excites the predetermined wavelength λx to the intensity and the excitation time of the excitation light that excites the predetermined wavelength λx under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if a detection intensity has been changed when the fluorescence image at the predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the predetermined wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed detection intensity to the detection intensity under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.
In the above-mentioned fluorescence-endoscope apparatus, if a gain has been changed when the fluorescence image at the predetermined wavelength λx among the wavelength λ1 to wavelength λn is acquired by the fluorescence image capturing unit, in accordance with the change, when the density D1 of the fluorescent component 1 to the density Dm of the fluorescent component m are calculated using Equation (1), the fluorescent-component-density computation unit may multiply the coefficients a1 (λx) to am(λx) at the predetermined wavelength λx of the fluorescent component 1 to fluorescent component m at the normalized density under the reference exposure conditions by the ratio of the changed gain to the gain under the reference exposure conditions when the fluorescence image at the predetermined wavelength λx is acquired by the fluorescence image capturing unit.